Label-driven weakly-supervised learning for multimodal deformable image registration

摘要

Spatially aligning medical images from different modalities remains achallenging task, especially for intraoperative applications that require fastand robust algorithms. We propose a weakly-supervised, label-driven formulationfor learning 3D voxel correspondence from higher-level label correspondence,thereby bypassing classical intensity-based image similarity measures. Duringtraining, a convolutional neural network is optimised by outputting a densedisplacement field (DDF) that warps a set of available anatomical labels fromthe moving image to match their corresponding counterparts in the fixed image.These label pairs, including solid organs, ducts, vessels, point landmarks andother ad hoc structures, are only required at training time and can bespatially aligned by minimising a cross-entropy function of the warped movinglabel and the fixed label. During inference, the trained network takes a newimage pair to predict an optimal DDF, resulting in a fully-automatic,label-free, real-time and deformable registration. For interventionalapplications where large global transformation prevails, we also propose aneural network architecture to jointly optimise the global- and localdisplacements. Experiment results are presented based on cross-validatingregistrations of 111 pairs of T2-weighted magnetic resonance images and 3Dtransrectal ultrasound images from prostate cancer patients with a total ofover 4000 anatomical labels, yielding a median target registration error of 4.2mm on landmark centroids and a median Dice of 0.88 on prostate glands.